Example Notebook for Correcting and loading FastCCD Images

This example reads data from the FastCCD and corrects and rotates the images. Included are some example of widgets to interact with the data

Getting Started

Load the databroker moudle, csxtools and various other dependencies

import numpy as np
from databroker import DataBroker, get_table, get_images
from csxtools.utils import get_fastccd_images
from csxtools.ipynb import image_stack_to_movie, show_image_stack
%matplotlib inline
from matplotlib import pyplot as plt

Set the logger level to info to see logging output

import logging
logging.getLogger().setLevel(logging.INFO)

Get the data!

Load the headers without the images (using fill=False)

bgnd = DataBroker[57512] #[52440]
print(bgnd)
get_table(bgnd, fill=False)
INFO:databroker.databroker:Interpreting key = 57512 as an integer
header
======

  EventDescriptor
  ---------------
  configuration   :
        ring_curr       : value
        ring_curr       : timestamp
          dtype           : number
          source          : PV:XF:23ID-SR{}I-I
          upper_ctrl_limit: 0.0
          lower_ctrl_limit: 0.0
          units           : mA
          shape           : []
          precision       : 2
        sclr_preset_time: value
        sclr_preset_time: timestamp
          dtype           : number
          source          : PV:XF:23ID1-ES{Sclr:1}.TP
          upper_ctrl_limit: 0.0
          lower_ctrl_limit: 0.0
          units           :
          shape           : []
          precision       : 3
  +--------------------------+---------+------------------+------------+---------------------+-------------+-----------+-----------------+--------------------------------------------+-------+------------------+
  | data keys                |  dtype  |    enum_strs     |  external  |   lower_ctrl_limit  | object_name | precision |      shape      |                   source                   | units | upper_ctrl_limit |
  +--------------------------+---------+------------------+------------+---------------------+-------------+-----------+-----------------+--------------------------------------------+-------+------------------+
  | fccd_acquire_time        |  number |                  |            |         0.0         |     fccd    |     3     |        []       |  PV:XF:23ID1-ES{FCCD}cam1:AcquireTime_RBV  |       |       0.0        |
  | fccd_image               |  array  |                  | FILESTORE: |                     |     fccd    |           | [1152, 2050, 0] |            PV:XF:23ID1-ES{FCCD}            |       |                  |
  | fccd_num_images_captured | integer |                  |            |                     |     fccd    |           |        []       |  PV:XF:23ID1-ES{FCCD}HDF1:NumCaptured_RBV  |       |                  |
  | fccd_plugin_num_images   |  number |                  |            |         0.0         |     fccd    |     0     |        []       | PV:XF:23ID1-ES{FCCD}FastCCD1:NumImages_RBV |       |       0.0        |
  | fccd_stats1_total        |  number |                  |            |                     |     fccd    |     0     |        []       |    PV:XF:23ID1-ES{FCCD}Stats1:Total_RBV    |       |                  |
  | fccd_stats2_total        |  number |                  |            |                     |     fccd    |     0     |        []       |    PV:XF:23ID1-ES{FCCD}Stats2:Total_RBV    |       |                  |
  | fccd_stats3_total        |  number |                  |            |                     |     fccd    |     0     |        []       |    PV:XF:23ID1-ES{FCCD}Stats3:Total_RBV    |       |                  |
  | fccd_stats4_total        |  number |                  |            |                     |     fccd    |     0     |        []       |    PV:XF:23ID1-ES{FCCD}Stats4:Total_RBV    |       |                  |
  | fccd_stats5_total        |  number |                  |            |                     |     fccd    |     0     |        []       |    PV:XF:23ID1-ES{FCCD}Stats5:Total_RBV    |       |                  |
  | m1a_pit_actuate          | integer |                  |            |          0          |     m1a     |           |        []       |    PV:XF:23IDA-OP:1{Mir:1}MOVE_CMD.PROC    |       |        0         |
  | m1a_pit_done             | integer | ['Done', 'Busy'] |            |                     |     m1a     |           |        []       |      PV:XF:23IDA-OP:1{Mir:1}BUSY_STS       |       |                  |
  | m1a_pit_readback         |  number |                  |            |                     |     m1a     |     3     |        []       |   PV:XF:23IDA-OP:1{Mir:1-Ax:Pit}Mtr_MON    |  mrad |                  |
  | m1a_pit_setpoint         |  number |                  |            |        -20.0        |     m1a     |     3     |        []       |  PV:XF:23IDA-OP:1{Mir:1-Ax:Pit}Mtr_POS_SP  |  mrad |       20.0       |
  | m1a_pit_stop_signal      | integer |                  |            |          0          |     m1a     |           |        []       |    PV:XF:23IDA-OP:1{Mir:1}STOP_CMD.PROC    |       |        0         |
  | m1a_rol_actuate          | integer |                  |            |          0          |     m1a     |           |        []       |    PV:XF:23IDA-OP:1{Mir:1}MOVE_CMD.PROC    |       |        0         |
  | m1a_rol_done             | integer | ['Done', 'Busy'] |            |                     |     m1a     |           |        []       |      PV:XF:23IDA-OP:1{Mir:1}BUSY_STS       |       |                  |
  | m1a_rol_readback         |  number |                  |            |                     |     m1a     |     3     |        []       |   PV:XF:23IDA-OP:1{Mir:1-Ax:Rol}Mtr_MON    |  mrad |                  |
  | m1a_rol_setpoint         |  number |                  |            |        -20.0        |     m1a     |     3     |        []       |  PV:XF:23IDA-OP:1{Mir:1-Ax:Rol}Mtr_POS_SP  |  mrad |       20.0       |
  | m1a_rol_stop_signal      | integer |                  |            |          0          |     m1a     |           |        []       |    PV:XF:23IDA-OP:1{Mir:1}STOP_CMD.PROC    |       |        0         |
  | m1a_x_actuate            | integer |                  |            |          0          |     m1a     |           |        []       |    PV:XF:23IDA-OP:1{Mir:1}MOVE_CMD.PROC    |       |        0         |
  | m1a_x_done               | integer | ['Done', 'Busy'] |            |                     |     m1a     |           |        []       |      PV:XF:23IDA-OP:1{Mir:1}BUSY_STS       |       |                  |
  | m1a_x_readback           |  number |                  |            |                     |     m1a     |     3     |        []       |    PV:XF:23IDA-OP:1{Mir:1-Ax:X}Mtr_MON     |   mm  |                  |
  | m1a_x_setpoint           |  number |                  |            |        -15.0        |     m1a     |     3     |        []       |   PV:XF:23IDA-OP:1{Mir:1-Ax:X}Mtr_POS_SP   |   mm  |       15.0       |
  | m1a_x_stop_signal        | integer |                  |            |          0          |     m1a     |           |        []       |    PV:XF:23IDA-OP:1{Mir:1}STOP_CMD.PROC    |       |        0         |
  | m1a_y_actuate            | integer |                  |            |          0          |     m1a     |           |        []       |    PV:XF:23IDA-OP:1{Mir:1}MOVE_CMD.PROC    |       |        0         |
  | m1a_y_done               | integer | ['Done', 'Busy'] |            |                     |     m1a     |           |        []       |      PV:XF:23IDA-OP:1{Mir:1}BUSY_STS       |       |                  |
  | m1a_y_readback           |  number |                  |            |                     |     m1a     |     3     |        []       |    PV:XF:23IDA-OP:1{Mir:1-Ax:Y}Mtr_MON     |   mm  |                  |
  | m1a_y_setpoint           |  number |                  |            |        -10.0        |     m1a     |     3     |        []       |   PV:XF:23IDA-OP:1{Mir:1-Ax:Y}Mtr_POS_SP   |   mm  |       10.0       |
  | m1a_y_stop_signal        | integer |                  |            |          0          |     m1a     |           |        []       |    PV:XF:23IDA-OP:1{Mir:1}STOP_CMD.PROC    |       |        0         |
  | m1a_yaw_actuate          | integer |                  |            |          0          |     m1a     |           |        []       |    PV:XF:23IDA-OP:1{Mir:1}MOVE_CMD.PROC    |       |        0         |
  | m1a_yaw_done             | integer | ['Done', 'Busy'] |            |                     |     m1a     |           |        []       |      PV:XF:23IDA-OP:1{Mir:1}BUSY_STS       |       |                  |
  | m1a_yaw_readback         |  number |                  |            |                     |     m1a     |     3     |        []       |   PV:XF:23IDA-OP:1{Mir:1-Ax:Yaw}Mtr_MON    |  mrad |                  |
  | m1a_yaw_setpoint         |  number |                  |            |        -20.0        |     m1a     |     3     |        []       |  PV:XF:23IDA-OP:1{Mir:1-Ax:Yaw}Mtr_POS_SP  |  mrad |       20.0       |
  | m1a_yaw_stop_signal      | integer |                  |            |          0          |     m1a     |           |        []       |    PV:XF:23IDA-OP:1{Mir:1}STOP_CMD.PROC    |       |        0         |
  | m1a_z_actuate            | integer |                  |            |          0          |     m1a     |           |        []       |    PV:XF:23IDA-OP:1{Mir:1}MOVE_CMD.PROC    |       |        0         |
  | m1a_z_done               | integer | ['Done', 'Busy'] |            |                     |     m1a     |           |        []       |      PV:XF:23IDA-OP:1{Mir:1}BUSY_STS       |       |                  |
  | m1a_z_readback           |  number |                  |            |                     |     m1a     |     3     |        []       |    PV:XF:23IDA-OP:1{Mir:1-Ax:Z}Mtr_MON     |   mm  |                  |
  | m1a_z_setpoint           |  number |                  |            |        -30.0        |     m1a     |     3     |        []       |   PV:XF:23IDA-OP:1{Mir:1-Ax:Z}Mtr_POS_SP   |   mm  |       30.0       |
  | m1a_z_stop_signal        | integer |                  |            |          0          |     m1a     |           |        []       |    PV:XF:23IDA-OP:1{Mir:1}STOP_CMD.PROC    |       |        0         |
  | m3a_bdr_user_readback    |  number |                  |            |                     |     m3a     |     4     |        []       |    PV:XF:23ID1-OP{Mir:3-Ax:Bdr}Mtr.RBV     |   mm  |                  |
  | m3a_bdr_user_setpoint    |  number |                  |            |        -10.0        |     m3a     |     4     |        []       |    PV:XF:23ID1-OP{Mir:3-Ax:Bdr}Mtr.VAL     |   mm  |       1.0        |
  | m3a_pit_user_readback    |  number |                  |            |                     |     m3a     |     4     |        []       |     PV:XF:23ID1-OP{Mir:3-Ax:P}Mtr.RBV      |  mrad |                  |
  | m3a_pit_user_setpoint    |  number |                  |            |         0.0         |     m3a     |     4     |        []       |     PV:XF:23ID1-OP{Mir:3-Ax:P}Mtr.VAL      |  mrad |       0.0        |
  | m3a_x_user_readback      |  number |                  |            |                     |     m3a     |     4     |        []       |    PV:XF:23ID1-OP{Mir:3-Ax:XAvg}Mtr.RBV    |   mm  |                  |
  | m3a_x_user_setpoint      |  number |                  |            |        -10.0        |     m3a     |     4     |        []       |    PV:XF:23ID1-OP{Mir:3-Ax:XAvg}Mtr.VAL    |   mm  |       10.0       |
  | ring_curr                |  number |                  |            |         0.0         |  ring_curr  |     2     |        []       |             PV:XF:23ID-SR{}I-I             |   mA  |       0.0        |
  | sclr_ch1                 |  number |                  |            |                     |     sclr    |     0     |        []       |         PV:XF:23ID1-ES{Sclr:1}.S1          |       |                  |
  | sclr_ch2                 |  number |                  |            |                     |     sclr    |     0     |        []       |         PV:XF:23ID1-ES{Sclr:1}.S2          |       |                  |
  | sclr_ch3                 |  number |                  |            |                     |     sclr    |     0     |        []       |         PV:XF:23ID1-ES{Sclr:1}.S3          |       |                  |
  | sclr_ch4                 |  number |                  |            |                     |     sclr    |     0     |        []       |         PV:XF:23ID1-ES{Sclr:1}.S4          |       |                  |
  | sclr_ch5                 |  number |                  |            |                     |     sclr    |     0     |        []       |         PV:XF:23ID1-ES{Sclr:1}.S5          |       |                  |
  | sclr_ch6                 |  number |                  |            |                     |     sclr    |     0     |        []       |         PV:XF:23ID1-ES{Sclr:1}.S6          |       |                  |
  | sclr_time                |  number |                  |            |         0.0         |     sclr    |     3     |        []       |          PV:XF:23ID1-ES{Sclr:1}.T          |       |       0.0        |
  | slt1_xc_user_readback    |  number |                  |            |                     |     slt1    |     4     |        []       |    PV:XF:23ID1-OP{Slt:1-Ax:XCtr}Mtr.RBV    |   mm  |                  |
  | slt1_xc_user_setpoint    |  number |                  |            |        -10.0        |     slt1    |     4     |        []       |    PV:XF:23ID1-OP{Slt:1-Ax:XCtr}Mtr.VAL    |   mm  |       10.0       |
  | slt1_xg_user_readback    |  number |                  |            |                     |     slt1    |     4     |        []       |    PV:XF:23ID1-OP{Slt:1-Ax:XGap}Mtr.RBV    |   mm  |                  |
  | slt1_xg_user_setpoint    |  number |                  |            | -30.000000000000004 |     slt1    |     4     |        []       |    PV:XF:23ID1-OP{Slt:1-Ax:XGap}Mtr.VAL    |   mm  |       30.0       |
  | slt1_yc_user_readback    |  number |                  |            |                     |     slt1    |     4     |        []       |    PV:XF:23ID1-OP{Slt:1-Ax:YCtr}Mtr.RBV    |   mm  |                  |
  | slt1_yc_user_setpoint    |  number |                  |            |        1.4003       |     slt1    |     4     |        []       |    PV:XF:23ID1-OP{Slt:1-Ax:YCtr}Mtr.VAL    |   mm  |      1.4003      |
  | slt1_yg_user_readback    |  number |                  |            |                     |     slt1    |     4     |        []       |    PV:XF:23ID1-OP{Slt:1-Ax:YGap}Mtr.RBV    |   mm  |                  |
  | slt1_yg_user_setpoint    |  number |                  |            | -20.000899999999998 |     slt1    |     4     |        []       |    PV:XF:23ID1-OP{Slt:1-Ax:YGap}Mtr.VAL    |   mm  |     19.9991      |
  | slt2_xc_user_readback    |  number |                  |            |                     |     slt2    |     4     |        []       |    PV:XF:23ID1-OP{Slt:2-Ax:XCtr}Mtr.RBV    |   mm  |                  |
  | slt2_xc_user_setpoint    |  number |                  |            |        -0.069       |     slt2    |     4     |        []       |    PV:XF:23ID1-OP{Slt:2-Ax:XCtr}Mtr.VAL    |   mm  |      -0.069      |
  | slt2_xg_user_readback    |  number |                  |            |                     |     slt2    |     4     |        []       |    PV:XF:23ID1-OP{Slt:2-Ax:XGap}Mtr.RBV    |   mm  |                  |
  | slt2_xg_user_setpoint    |  number |                  |            |       14.2938       |     slt2    |     4     |        []       |    PV:XF:23ID1-OP{Slt:2-Ax:XGap}Mtr.VAL    |   mm  |     14.2938      |
  | slt2_yc_user_readback    |  number |                  |            |                     |     slt2    |     4     |        []       |    PV:XF:23ID1-OP{Slt:2-Ax:YCtr}Mtr.RBV    |   mm  |                  |
  | slt2_yc_user_setpoint    |  number |                  |            |        0.1515       |     slt2    |     4     |        []       |    PV:XF:23ID1-OP{Slt:2-Ax:YCtr}Mtr.VAL    |   mm  |      0.1515      |
  | slt2_yg_user_readback    |  number |                  |            |                     |     slt2    |     4     |        []       |    PV:XF:23ID1-OP{Slt:2-Ax:YGap}Mtr.RBV    |   mm  |                  |
  | slt2_yg_user_setpoint    |  number |                  |            |       14.6062       |     slt2    |     4     |        []       |    PV:XF:23ID1-OP{Slt:2-Ax:YGap}Mtr.VAL    |   mm  |     14.6062      |
  | slt3_x_user_readback     |  number |                  |            |                     |     slt3    |     4     |        []       |     PV:XF:23ID1-OP{Slt:3-Ax:X}Mtr.RBV      |   mm  |                  |
  | slt3_x_user_setpoint     |  number |                  |            |       -1000.0       |     slt3    |     4     |        []       |     PV:XF:23ID1-OP{Slt:3-Ax:X}Mtr.VAL      |   mm  |      1000.0      |
  | slt3_y_user_readback     |  number |                  |            |                     |     slt3    |     4     |        []       |     PV:XF:23ID1-OP{Slt:3-Ax:Y}Mtr.RBV      |   mm  |                  |
  | slt3_y_user_setpoint     |  number |                  |            |         0.0         |     slt3    |     4     |        []       |     PV:XF:23ID1-OP{Slt:3-Ax:Y}Mtr.VAL      |   mm  |       0.0        |
  | temp_a                   |  number |                  |            |                     |     temp    |     3     |        []       |     PV:XF:23ID1-ES{TCtrl:1-Chan:A}T-I      |   K   |                  |
  | temp_b                   |  number |                  |            |                     |     temp    |     3     |        []       |     PV:XF:23ID1-ES{TCtrl:1-Chan:B}T-I      |   K   |                  |
  +--------------------------+---------+------------------+------------+---------------------+-------------+-----------+-----------------+--------------------------------------------+-------+------------------+
  run_start       : b636d0b5-28be-4280-a7e7-01e76fdb30f8
  time            : 1455378251.731183
  uid             : aa53ecdf-a1c4-40a1-9fe6-bf0aac32a692

  RunStart
  --------
  beamline_id     : CSX-1
  config          :
  detectors       : ['temp', 'slt1', 'slt2', 'slt3', 'm1a', 'm3a', 'ring_curr', 'sclr', 'fccd']
  group           :
  owner           : xf23id1
  plan_args       :
    detectors       : [Temperature(prefix='XF:23ID1-ES{TCtrl:1', name='temp', read_attrs=['a', 'b'], configuration_attrs=[], monitor_attrs=[]), SlitsGapCenter(prefix='XF:23ID1-OP{Slt:1', name='slt1', read_attrs=['xg', 'xc', 'yg', 'yc'], configuration_attrs=[], monitor_attrs=[]), SlitsGapCenter(prefix='XF:23ID1-OP{Slt:2', name='slt2', read_attrs=['xg', 'xc', 'yg', 'yc'], configuration_attrs=[], monitor_attrs=[]), SlitsXY(prefix='XF:23ID1-OP{Slt:3', name='slt3', read_attrs=['x', 'y'], configuration_attrs=[], monitor_attrs=[]), Mirror(prefix='XF:23IDA-OP:1{Mir:1', name='m1a', read_attrs=['z', 'y', 'x', 'pit', 'yaw', 'rol'], configuration_attrs=[], monitor_attrs=[]), MotorMirror(prefix='XF:23ID1-OP{Mir:3', name='m3a', read_attrs=['x', 'pit', 'bdr'], configuration_attrs=[], monitor_attrs=[]), EpicsSignal(read_pv='XF:23ID-SR{}I-I', name='ring_curr', value=175.2972070250072, timestamp=1455378190.755994, pv_kw={}, auto_monitor=None, string=False, write_pv='XF:23ID-SR{}I-I', limits=False, put_complete=False), EpicsScaler(prefix='XF:23ID1-ES{Sclr:1}', name='sclr', read_attrs=['channels', 'time'], configuration_attrs=['preset_time', 'presets', 'gates'], monitor_attrs=[]), ProductionCam(prefix='XF:23ID1-ES{FCCD}', name='fccd', read_attrs=['hdf5', 'acquire_time', 'num_images_captured', 'plugin_num_images', 'stats1', 'stats2', 'stats3', 'stats4', 'stats5'], configuration_attrs=[], monitor_attrs=[])]
    delay           : 0
    num             : 1
  plan_type       : Count
  project         : Prop. 300121, SAF = 300275
  sample          :
    type            : small
    composition     : LBCO 1/8
  scan_id         : 57512
  time            : 1455378191.0151987
  uid             : b636d0b5-28be-4280-a7e7-01e76fdb30f8

  RunStop
  -------
  exit_status     : success
  reason          :
  run_start       : b636d0b5-28be-4280-a7e7-01e76fdb30f8
  time            : 1455378251.826415
  uid             : 01ad412f-59e5-4ff1-84bf-dca265ca3f77
time m3a_x_user_setpoint slt1_xg_user_setpoint slt3_y_user_readback slt1_yg_user_readback slt2_yg_user_setpoint m3a_pit_user_setpoint slt3_x_user_readback m1a_yaw_readback m1a_y_actuate ... m1a_yaw_setpoint slt2_yc_user_readback m1a_y_readback slt2_xg_user_readback slt2_yg_user_readback fccd_stats1_total m1a_z_setpoint m1a_rol_done sclr_time m1a_x_readback
1 2016-02-13 15:44:11.782440-05:00 0 5 -0.25073 5.0001 -1.776357e-15 -0.616666 -6.0802 0.000091 1 ... 0 -5.0001 -2.409994 10 -1.776357e-15 1451 -27.62 0 1 0.000024

1 rows × 76 columns

data = DataBroker[57511] #[52436]
get_table(data, fill=False)
INFO:databroker.databroker:Interpreting key = 57511 as an integer
time m3a_x_user_setpoint slt1_xg_user_setpoint slt3_y_user_readback slt1_yg_user_readback slt2_yg_user_setpoint m3a_pit_user_setpoint slt3_x_user_readback m1a_yaw_readback m1a_y_actuate ... m1a_yaw_setpoint slt2_yc_user_readback m1a_y_readback slt2_xg_user_readback slt2_yg_user_readback fccd_stats1_total m1a_z_setpoint m1a_rol_done sclr_time m1a_x_readback
1 2016-02-13 15:40:56.780111-05:00 0 5 -0.25073 5.0002 -1.776357e-15 -0.616666 -6.08021 0.000091 1 ... 0 -5.0001 -2.409994 10 -1.776357e-15 1360683 -27.62 0 1 0.000024

1 rows × 76 columns

Correct the FastCCD Images

Now correct the CCD images, using only a single background image.

In the future this should be taken from the metadata associated with the data, or from the darkfield entry

First lets just look at the background stack as a “lightfield”

images = get_fastccd_images(data, (bgnd, None, None), tag='fccd_image')
INFO:csxtools.utils:Took 1.64s to read data using get_images
INFO:csxtools.fastccd.images:Correcting image stack of shape (600, 960, 960)
INFO:csxtools.fastccd.images:Not correcting for darkfield. No input.
INFO:csxtools.fastccd.images:Not correcting for flatfield. No input.
INFO:csxtools.fastccd.images:Corrected image stack in 0.359s
WARNING:csxtools.utils:Missing dark image for gain setting 1
WARNING:csxtools.utils:Missing dark image for gain setting 2
INFO:csxtools.utils:Computed dark images in 17.6s
INFO:csxtools.utils:Took 28.7s to read data using get_images
INFO:csxtools.fastccd.images:Correcting image stack of shape (6000, 960, 960)
INFO:csxtools.fastccd.images:Not correcting for flatfield. No input.
(1, 600, 960, 960) (3, 960, 960) (960, 960)
(1, 6000, 960, 960)
INFO:csxtools.fastccd.images:Corrected image stack in 3.31s
(3, 960, 960) (960, 960)
images.shape
(1, 6000, 960, 960)

Now lets do an interactive plotting widget!

Thanks ipywidgets!

show_image_stack(np.log10(images[0]), (-10, 10000))
../_images/Correct_FastCCD_Images-Copy1_12_0.png

And lets do a movie of the stack of images

This produces an HTML5 movie which can be viewed in most browsers

image_stack_to_movie(images[0], frames=100, figsize=(12, 10), vmin=-20, vmax=300)

What version are we running?

import csxtools
csxtools.__version__
images.shape
(1, 6000, 960, 960)
plt.imshow(np.log10(np.nanmean(images[0][0:100,400:480,520:630], axis=0)), vmin=-3, vmax=3)
<matplotlib.image.AxesImage at 0x7fb3ed7121d0>
../_images/Correct_FastCCD_Images-Copy1_18_1.png
stack =
---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-30-a30db1dae88d> in <module>()
----> 1 stack = np.log10(np.nanmean(images[0][0:100,400:480,520:630], axis=0), vmin=0, vmax=1)


TypeError: 'vmax' is an invalid keyword to ufunc 'log10'
np.nanmax(stack)
3.9225066
800/(2.8*6)
47.61904761904763